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1  Project  Objectives 

The  research  objectives  of  the  project  revolve  around  the  development  of  a  rigorous  mathematical 
framework  for  uncertainty-based  design  optimization,  with  an  emphasis  on  the  construction  and 
analysis  of  a  set  of  efficient  numerical  methods  applicable  for  complex  systems.  The  objectives 
include  the  following  major  components: 

1.  Constructing  a  mathematical  framework,  using  generalized  polynomial  chaos  (gPC)  theory, 
to  facilitate  the  analysis  of  uncertainty-based  design  optimization. 

2.  Developing  novel  and  efficient  numerical  algorithms. 

3.  Conducting  rigorous  analysis  and  error  estimates  for  the  new  algorithms. 

The  work  is  performed  on  two  kinds  of  uncertainty-based  design  optimization: 

•  Robust  design  optimization,  where  the  objective  is  to  optimize  the  system  performance  while 
minimizing  its  sensitivity  to  uncertainty. 

•  Reliability-based  design  optimization,  where  the  objective  is  to  optimize  the  system  perfor¬ 
mance  while  keeping  its  failure  probability  under  control. 


2  Major  Achievements 

We  report  here  that  we  have  made  significant  progresses  and  accomplished  notable  achievements  in 
all  fronts  of  the  project  objectives.  To  facilitate  the  discussion,  we  employ  the  following  notation: 
let  a:  be  a  set  of  design  variables  for  a  system  S,  and  Z  be  a  set  of  random  variables  representing 
uncertainty  in  the  system,  and  f(x,  Z )  be  a  cost  function  to  be  minimized.  The  underlying  assump¬ 
tion  is  that  the  system  S  highly  complex,  and  consequently  the  cost  function  /  is  time  consuming 
to  simulate. 

2.1  Robust  Design  Optimization 

In  robust  optimization  the  goal  is  to  minimize  the  cost  function  and  simultaneously  its  sensitivity 
to  the  uncertainty  input.  The  most  widely  adopted  formulation  takes  the  following  form 

min +  a  ■  Vf(x)),  (1) 

where  /Jj  is  the  mean  of  the  cost  function  and  Vj  its  variance,  a  is  a  parameter  specifying  the 
significance  of  the  variance  in  this  formulation.  It  is  clear  that  this  formulation  seeks  to  minimize 
both  the  average  behavior  of  the  cost  function  and  its  variance. 
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2.1.1  Theoretical  Achievement 

It  was  proposed  in  this  project  that  the  traditional  formulation  (1)  is  not  sufficient  to  guarantee 
that  the  optimal  solution  is  “least  sensitive”.  This  is  because  the  use  of  variance  Vf,  which  is  a 
weak  measure  of  variability.  For  example,  the  objective  functions  shown  in  Fig.  1  all  have  the  same 
variance,  but  their  sensitivity  towards  randomness  are  obviously  drastically  different. 


z  z 


Fig.  1:  Illustrative  examples  of  objective  functions  /  in  term  of  random  parameter  Z.  All  responses 
fi(Z),i  =  1,2,3, 4  have  same  mean  (~  0)  and  STD  a  ~  0.3,  but  the  “sensitiveness”  with  respect 
to  Z  is  drastically  different.  f\  is  obviously  the  most  “robust”  choice. 

To  circumvent  the  difficulty,  a  set  of  “strong”  measures  for  robustness  are  proposed  in  [4], 
Instead  of  defining  Vf  by  the  variance  as  in  (1),  we  define 

Vf  =  E 

where  the  p-norrn  can  be  the  standard  oo-norm, 


df 

dZ 

p. 

(2) 


1-norm,  and  2-norm.  These  measures  can  more 


Measure 

h 

h 

/3 

U 

STD 

0.3 

0.3 

0.3 

0.3 

p  =  0 

2.4 

20.0 

65.7 

111.1 

P  =  2 

1.6 

5.3 

19.6 

14.7 

Table  1:  Different  measures  of  the  variability  of  the  illustrative  response  functions  in  Fig.  1,  using 
the  p-norrn  from  (2). 


accurately  reflect  the  variability  of  a  given  response  function.  And  this  is  demonstrated  by  applying 
these  measures  to  the  illustrative  functions  fi,  f 2,  f 3,  and  in  Fig.  1.  The  results  are  summarized 
in  Table  1.  Clearly,  the  “least  robust”  objective  function  and  are  easily  identified. 

2.1.2  Development  of  Efficient  Algorithms 

A  set  of  highly  efficient  numerical  algorithms  were  developed  using  the  theory  of  generalized  polyno¬ 
mial  chaos  (gPC),  for  both  the  traditional  weak  measure  of  robustness  and  the  new  strong  measure 
of  robustness.  The  gPC  theory,  an  extension  of  the  earlier  work  of  polynomial  chaos  (PC)  [1],  is 
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first  proposed  in  [5].  It  is  an  orthogonal  polynomial  approximation  in  multi-dimensional  space.  In 
gPC,  the  cost  function  can  be  approximated  by 


N 

f(x,  Z)  «  fN(x,  Z)  =  (3) 

j= o 

where  a3  are  the  coefficients  and  <!>_,■  are  orthogonal  polynomials  satisfying 

E  [$i{Z)$j(Z)]  =  8ij,  (4) 

where  the  operator  E[-]  stands  for  expectation  (or,  mean,  average).  Due  to  the  orthogonality,  we 
have 

N 

Hf  =  o0(x),  Vf  =  Y1  (5) 

3= 1 

The  robust  optimization  problem  (1)  can  then  be  reformulated  entirely  in  term  of  the  gPC  expansion 
coefficients.  And  all  other  statistical  quantities,  including  those  required  by  the  strong  robustness 
measure  (2),  can  be  easily  evaluated  by  the  gPC  approximation  without  simulating  the  system. 
For  examples,  let  the  objective  function  to  be 

f(u,w)  =  \  f  ( u-ud)2dx+ -  [  (w-wd)2dx  (6) 

z  Jn  1  Jon 

subject  to  a  stochastic  diffusion  equation 

Lu(x,z)  =  f(x),  xGD  , 

u  =  w,  x  E  d£l.  v 

A  representative  result  is  shown  in  Fig.  2.  We  observe  significant  efficiency  gain  obtained  by 

stochastic  collocation  (SC)  based  gPC  solver. 


Fig.  2:  Comparison  of  the  solution  of  (7)  between  Monte  Carlo  (MC)  method  and  stochastic 
collocation  (SC)  gPC  method.  Left:  standard  deviation;  right:  zoomed  view  of  standard  deviation. 
The  lst-order  SC  requires  41  samples  and  the  2nd-order  requires  841  samples.  Both  are  much 
smaller  than  10,000  MC  samples.  Details  are  in  [4]. 
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2.2  Reliability-based  Design  Optimization 

In  a  very  general  setting,  the  basic  formulation  for  reliability-based  design  optimization  (RBDO) 
takes  the  following  form. 


min  f(x,Z), 

X 

subject  to  Pf  <  e, 


(8) 


where  Pf  is  the  failure  probability  of  the  system  and  e  >  0  is  a  small  number.  The  failure  probability 
is  defined  as 

Pf  =  Prob \g(Z)  <  0]  =  [  p{z)dz,  (9) 

Jg(z)<0 

where  p(z)  is  the  probability  distribution  of  the  random  variables  Z,  and  g  is  the  so-called  “limit 
state  function”  or  “failure  function”  of  the  system  S.  The  design  space  is  separated  into  two  regions 
by  g  =  0,  the  safe  region  g  >  0  and  the  failure  region  g  <  0. 


2.2.1  Theoretical  Achievement 


The  key  component  in  RBDO  is  the  evaluation  of  the  failure  probability  Pf  in  (9).  The  most 
reliable  method  is  the  Monte  Carlo  sampling  (MCS),  where  the  failure  probability  is  estimated  by 


rymc 


Number  of  samples  in  failure  domain 
Total  number  of  samples 


(10) 


Though  simple  to  implement,  the  method  poses  numerical  challenge.  It  requires  about  at  least 
10  failed  samples  to  reliably  estimate  the  failure  probability.  This  usually  translates  into  a  large 
number  of  total  samples.  For  example,  for  Pf  ~  10-5,  it  is  quite  common  to  require  106  samples. 
Even  with  the  most  sophisticated  important  sampling  (IS)  method,  a  minimal  of  104  samples  are 
required.  For  any  practical  large-scale  systems,  the  required  number  of  samples  is  completely  out 
of  reach. 

In  this  project,  we  have  developed  a  novel  method  to  estimate  failure  probability.  The  key 
features  of  the  new  method  include 


•  Ftigh  accuracy.  The  method  is  accurate  and  does  not  have  any  approximation.  It  can  repro¬ 
duce  exactly  the  result  obtained  by  traditional  MCS.  In  many  examples  the  new  method  with 
~  102  samples  can  match  every  single  digit  of  the  MCS  results  obtained  by  ~  106  samples. 

•  Rigorousness.  The  theoretical  foundation  is  established  rigorously  by  our  convergence  theo¬ 
rem  in  [3]. 


2.2.2  Development  of  Efficient  Algorithms 

To  successfully  implement  the  new  method  for  practical  complex  system,  we  developed  a  highly 
efficient  predictor-correction  algorithm.  The  feature  of  the  algorithm  includes 

•  Unparalleled  efficiency.  This  method  allows  us  to  compute  rare  failure  probability  as  low  as 
Pf  10  6  with  only  a  few  hundred  samples.  This  capability  is  superior  than  any  of  the 
existing  methods  in  the  world  by  several  orders. 

•  Ease  of  implementation.  The  coding  effort  of  the  new  algorithm  is  close  to  zero.  One  can 
easily  adopt  the  algorithm  for  complex  systems  which  are  simulated  by  legacy  codes. 
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A  typical  example  is  shown  in  Table  2.  Here  the  state-of-the-art  importante  sampling  (IS) 
method,  the  cross-entropy  method,  is  employed.  And  it  produces  rare  failure  probability  more 
accurate  than  the  brute-force  MC  and  uses  only  25, 000  samples.  Our  new  algorithm  can  exactly 
reproduce  this  accurate  result  with  only  400  samples!  Extensive  numerical  tests  were  conducted  for 
a  variety  of  problems  and  all  results  are  very  similar.  The  new  method  makes  accurate  computation 
of  rare  failure  probability  possible  for  practical  systems. 


MC 

MC+IS 

New  method 

Pf 

2.02  x  10_e 

2.025  x  10-6 

2.025  x  10-6 

ff  of  samples 

108 

25,000 

400 

Table  2:  Example  of  computing  rare  failure  probability.  The  Important  sampling  (MC+IS)  pro¬ 
duces  a  better  result  than  MC  and  requires  25, 000  samples.  Our  new  method  can  reproduce  exactly 
the  same  result  with  only  400  samples.  Details  are  in  [3]. 


2.3  Other  Aspects:  Epistemic  Uncertainty 

Other  notable  accomplishments  made  during  the  project  include  the  development  of  a  computa¬ 
tional  strategy  for  epistemic  uncertainty  analysis.  Epistemic  uncertainty  refers  to  the  uncertainty 
without  sufficient  knowledge  to  specify  its  probability.  And  this  is  frequently  encountered  in  design 
and  optimization.  Owing  to  its  difficulty,  there  has  been  no  systematical  study.  In  this  project,  we 
have  produced  the  very  first  systematical  numerical  framework  to  quantify  epistemic  uncertainty. 
This  work,  published  in  [2],  consists  of  the  following  major  steps. 

•  Encapsulation  of  the  epistemic  uncertainty.  Here  a  bound  of  the  epistemic  uncertainty  is  esti¬ 
mated  so  that  all  epistemic  uncertainty  can  be  encapsulated  in  a  sufficiently  large  hypercube. 

•  Solution  of  the  governing  PDE  in  the  hypercube.  Here  no  probability  distribution  is  imposed. 

•  Post-processing  the  result.  If  a  posterior  probability  information  about  the  epistemic  uncer¬ 
tainty  is  available,  we  can  then  post-process  the  simulation  result  to  obtain  better  under¬ 
standing  of  the  solution. 

An  example  is  shown  in  Fig.  3.  This  is  a  result  of  an  oscillator,  which  is  an  ODE  with  six 
parameters.  Two  of  the  parameters  are  epistemic  variables.  The  results  here  show  the  CDF  of 
the  final  solution.  Note  the  CDF  is  a  function  of  the  two  epistemic  variables,  and  here  we  plot  25 
ensembles  of  the  CDF. 

This  work  will  certainly  lead  the  way  of  the  study  of  epistemic  uncertainty  analysis  and  play  a 
key  role  in  design  and  optimization  under  uncertainty. 

3  Student  support 

During  the  course  of  the  project,  the  research  fund  is  used  to  support  one  graduate  student,  Ms 
Jing  Li.  Since  the  fund  is  for  one  semester  and  summer  each  year,  other  funds  (e.g.  from  NSF)  is 
used  to  leverage  the  support. 

4  Other  Synthetic  Activities 

The  PI  has  attended  almost  all  major  conferences  and  workshops  on  UQ,  and  presented  numerous 
invited  lectures.  Notable  ones  include 
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Fig.  3:  An  ensemble  of  25  CDFs  of  the  solution  to  the  damped  harmonic  oscillator  subject  to 
mixed  aleatory  and  epistemic  uncertainty.  Each  CDF  corresponds  to  one  particular  set  of  epistemic 
variables  values. 

•  Plenary  Talk,  High  Order  Methods  for  Stochastic  Computations,  International  Conference  On 
Spectral  and  High  Order  Methods  (ICOSAHOM07),  Chinese  Academy  of  Sciences,  Beijing, 
China,  June  18-22,  2007. 

•  Plenary  talk.  Ecole  Doctorale  sur  la  Quantification  des  Incertitudes,  Paris,  November  24-27, 
2008. 

•  Plenary  talk.  The  Third  International  Conference  on  Scientific  Computing  and  Partial  Dif¬ 
ferential  Equations,  Hong  Kong,  December  8-13,  2008. 

•  Invited  speaker.  Workshop  on  Numerical  Solution  of  Stochastic  Partial  Differential  Equations, 
Dipartimento  di  Matematica,  Politecnico  di  Torino,  Italy,  May  10-13,  2010. 

•  Invited  speaker.  Workshop  on  Uncertainty  Quantification,  The  Royal  Society  of  Edinburgh, 
Scotland,  May  24-28,  2010. 

•  Colloquium  and  seminar.  University  of  Buffalo,  Arizona  State  University,  Stanford  Univer¬ 
sity,  Lawrence  Livermore  National  Lab,  George  Mason  University,  Northwestern  University, 
University  of  Southern  California,  Brown  University,  etc. 

Also,  the  PI  has  become  the  first  Associate  editor-in-chief  for  the  first  journal  dedicated  to  UQ, 
the  International  Journal  for  Uncertainty  Quantification. 

5  Publications 

Under  the  grant,  the  following  papers  have  been  published. 

1.  D.  Xiu,  “Fast  Numerical  Methods  for  Robust  Optimal  Design”,  Engineering  Optimization , 
Vol.  40,  489-504  2008. 

2.  D.  Xiu  and  J.  Shen,  “Efficient  Stochastic  Galerkin  Methods  for  Random  Diffusion  Equations”, 
Journal  of  Computational  Physics ,  Vol.  228,  266-281,  2009. 
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3.  R.  Archibald,  A.  Gelb,  R.  Saxena  and  D.  Xiu,  “Discontinuity  Detection  in  Multivariate  Space 
for  Stochastic  Simulations”,  Journal  of  Computational  Physics,  Vol.  228,  2676-2689,  2009. 

4.  Y.  Marzouk  and  D.  Xiu,  “A  Stochastic  Collocation  Approach  to  Bayesian  Inference  in  Inverse 
Problems”,  Communications  in  Computational  Physics,  Vol.  6,  826-847,  2009. 

5.  J.  Li  and  D.  Xiu,  “A  Generalized  Polynomial  Chaos  Based  Ensemble  Kalman  Filter  with 
High  Accuracy”,  Journal  of  Computational  Physics,  Vol.  228,  5454-5469,  2009. 

6.  J.  Jakeman,  M.  Eldred  and  D.  Xiu,  “Numerical  Approach  for  Quantification  of  Epistemic 
Uncertainty”,  Journal  of  Computational  Physics,  Vol.  229,  4648-4663,  2010. 

7.  Jing  Li  and  D.  Xiu,  “Evaluation  of  Failure  Probability  via  Surrogate  Models”,  Journal  Com¬ 
putational  Physics,  Vol.  229,  8966-8980,  2010. 

8.  P.  Tsuji,  D.  Xiu  and  L.  Ying,  “A  Fast  Method  for  High-frequency  Acoustic  Scattering  from 
Random  Scatterers”,  International  Journal  for  Uncertainty  Quantification,  Vol.  1,  99-117, 
2011. 

The  PI  also  published  the  first-ever  graduate-level  textbook  on  stochastic  simulation  and  un¬ 
certainty  quantification. 

•  D.  Xiu,  Numerical  Methods  for  Stochastic  Computations:  a  Spectral  Method  Approach,  Prince¬ 
ton  University  Press,  July  21,  2010.  Hard  cover:  152  pages.  ISBN-10:  0691142122,  ISBN-13: 
978-0691142128. 
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